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Background 

Research shows correlations between children’s problematic behaviours and reduced 
academic performance, stressed teacher-child relationships, poorer peer interactions, higher 
involvement in deviant behaviours, lower school adjustment, and increased chances of 
dropping out of school (Baker, Clark, Crowl, & Carlson, 2009; McMahon, Wells, & Kotler, 
2006). Problematic behaviour is a source of stress for teachers and predictive of teacher 
burnout (Kokkinos, 2007). 

Children use more prosocial behaviours in classrooms with teachers who are trained 
in positive management strategies compared to those whose teachers are not trained (Girard, 
Girolametto, Weitzman, & Greenberg, 2011). Developing teachers’ positive practices also 
benefits the mental health of educators, providing a community for those who feel isolated 
(Gebbie, Ceglowski, Taylor, & Miels, 2012). 

The Incredible Years (IY) Teacher Classroom Management (TCM) Programme is one 
intervention developed to address problematic behaviours via training teachers to use positive 
and proactive management strategies (Webster-Stratton, 2001). Ultimately, by changing 
teachers’ classroom management strategies, the programme aims to affect children’s 
behaviours. 


Research questions and design 
This systematic review utilises advanced statistical analyses and cross-synthesises evidence 
from both quantitative trials and qualitative studies to answer the research question: What is 
known about the effectiveness of the Incredible Years Teacher Classroom Management (TY 
TCM) Programme and how do people experience/perceive the programme and its effects? 
To answer this overarching question, there are two strands of sub-questions that first are 
addressed individually before being combined (Figure 1). 

To this end, this mixed methods systematic review can be divided into three phases: 
two independent and yet concurrently conducted strands for the quantitative (multilevel meta- 
analysis) and qualitative evidence (qualitative meta-synthesis) and one final cross-synthesis 
to connect these two strands (Figure 2). 


Methods 
A protocol for this review was submitted through the PROSPERO database. 


Studies included in the quantitative strand of this review must have met the following criteria: 
1. Population: early childhood and primary school teachers of children ages three- to 
eight-years-old 
2. Intervention: TY TCM 
3. Comparison: treatment-as-usual or waitlist control 
4. Design: RCTs 


The primary quantitative outcomes for this review were: 

1. Teacher classroom management strategies 

2. Child conduct problems 
A post hoc child behaviour difficulties analysis was added to also include measures of 
hyperactivity and peer problems. 


Secondary outcomes included: 
1. Teacher collaboration with parents 
2. Child prosocial behaviours 
3. Child academic readiness 
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Studies included in the qualitative strand of this review reported qualitative (1.e., non- 
numerical) data related to the TY TCM Programme. 
1. Data from qualitative studies or from mixed methods studies reporting qualitative 
outcomes; 
2. Studies clearly state that the participants’ responses pertain to [YY TCM; 
3. Those studies that combined IY TCM with additional programmes were included so 
long as data specific to TY TCM were reported separately. 


This review searched electronic databases, websites related to the IY Series, and reference 
lists of included studies (Figure 3). The search also included studies recommended by 
researchers in the field. Initial search terms were pared down to the most parsimonious set 
without compromising sensitivity (Figure 4). Two independent reviewers screened studies 
for relevance. 


Quantitative data were extracted into Microsoft Excel 2010. Studies were assessed using the 
Cochrane ‘Risk of Bias’ tool. Applying multilevel analysis techniques to a meta-analysis 
allows researchers to approximate the overall effect size (grand mean) on an outcome using 
all of the relevant outcome data available. Using the raw data reported in each included 
study, adjustments were first made according to the above ICC equations for the clustering of 
participants within studies before calculating the Hedges’ g effect sizes and running these in 
the multilevel meta-analysis to calculate the Hedges’ g grand mean (f9). Statistical analyses 
were run in R Studio (Version 3.2.2) using the metafor package (Viechtbauer, 2010). 


Qualitative data was extracted into NVivo 11 for analysis. Analysis was based in a grounded 
theory approach, which operates both inductively and deductively, and aims to generate, not 
test, theory (Glaser & Strauss, 1967). 


Findings from each strand were cross-synthesised using framework analysis and an 
integrative grid (Flemming, 2009). 


Results 
See Figure 5 for PRISMA flowchart. Fourteen records were included in this review and 
jointly described nine studies that fit into the quantitative, qualitative, or both sections of this 
paper. The nine included studies took place across five different countries: one study in 
England and two studies each in Ireland, Jamaica, the United States, and Wales. 


Effects of interventions (multilevel meta-analysis) 
See Table | and Figures 6-13 for a summary of the effects of interventions. There was a 
small, statistically significant effect of the [YY TCM intervention on reducing teacher negative 
classroom management strategies, child conduct problems, and high-risk child conduct 
problems. There was a moderate, statistically significant effect of the [YY TCM intervention 
on increasing teacher positive classroom management strategies. None of the included 
studies reported data on teacher collaboration with parents. There was no statistically 
significant effect of the [YY TCM intervention on increasing child prosocial behaviours or 
high-risk child prosocial behaviours. Only one study reported on child academic readiness, 
so a multilevel meta-analysis was not possible for this outcome. 

See Table 2 and Figures 14-16 for a summary of these post hoc multilevel meta- 
analysis results. There was a small, statistically significant effect of the [Y TCM intervention 
on reducing child behaviour difficulties and high-risk child behaviour difficulties. 
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Stakeholder experiences (qualitative meta-synthesis) 

Analysing the findings from each included study in this qualitative strand produced an 
overarching understanding of how key stakeholders (predominantly classroom teachers) 
experience the TY TCM Programme (Figure 17). What emerged was a cyclical process of 
engaging in the training and observing effective outcomes. Throughout the course, the 
teachers cycled between primary and secondary learning opportunities. Ultimately, this led 
to their experiences of the effects of [YY TCM training, and outcomes described by 
stakeholders are explained as they pertain to teachers, classrooms, and children (Figures 18 
and 19). 


Conclusions 
Decisions about which programmes to implement in schools are based on a variety of factors, 
including effectiveness of a given programme and how acceptable/appropriate it is for the 
context. By systematically reviewing effectiveness trial evidence alongside data on 
stakeholders’ experiences with the [YY TCM Programme, and then cross-sysnthesising the 
findings from both strands, this study provides the most comprehensive picture for both 
decision-makers and researchers in understanding if the TY TCM Programme benefited 
children and teachers and the processes of how people experience it. 
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Quantitative Strand: 
What are the effects of the 


IY TCM Programme on 
teachers’ management 
strategies and children’s 
behaviour, compared to 
treatment-as-usual, at post- 
intervention and any 
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points? 
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What are the experiences 
and views of key 
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Cross-synthesis: 
When considered in combination, what can the effectiveness and 


stakeholders’ experience data tell us? 
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Figure 1. Research questions 
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Cross-synthesis: 
* Framework analysis accompanied by an integrative grid to chart data 


* Narrative synthesis of findings 
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Figure 2. Research design and methods 
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Databases: 


L Campbell Collaboration Library 
http://www.campbellcollaboration.org /lib, 


i 

2. 

re 
10. 
11. 
12. 
3. 


2 Cochrane Collaboration Library 
http://www.cochranelibrary.com/cochrane-database-of-systematic- 


reviews/ 


Incredible Years Series Library http://incredibleyears.com/research- 
library/ 


Centre for Evidence-Based Early Intervention 
https://www.bangor.ac.uk/psychology/cebei/publications.php.en 


Figure 3. List of databases and websites searched 
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“teacher classroom management”.af. 
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“teacher training”.af. 


Figure 4. Search terms 
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Records identified through Additional records identified 
database searching through other sources 
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Figure 5. PRISMA flowchart 
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Table 1. Multilevel meta-analysis results for primary and secondary outcomes 


Outcome N(k) Bo SE 95% Cl 


Primary 

Teacher 7(3) -0.32* 0.15 (-0.62, -0.01) L1: 0.00 : 4.09 (0.66) 
negative L2: 0.00 

strategies 

Teacher 0.16 (0.13, 0.74) : : 5.16 (0.52) 
positive 

strategies 

Child 0.09 (-0.38, -0.01) 

conduct 

problems 

Sensitivity 0.10 (-0.32, 0.09) : : 5.95 (0.55) 


High risk 8(3)  -0.33*** 0.07 (-0.47, -0.18) ; 4.55 (0.71) 
subgroup 
Secondary 
Child 
prosocial 
skills 
Sensitivity 4(3) 0.13* 0.06 — (0.02, 0.24) L1: 0.00 3.43 (0.33) 
L2: 0.00 
High risk 4(3) 0.17 (-0.03, 0.65) L1: 0.00 8.23 (0.04) 
subgroup L2: 0.00 
N=number of effect sizes (Level 1); k=number of studies (Level 2); By= Hedges’ g (‘Grand Mean); 
*n<0.05, **p<0.01, ***p<0.001 


Study Outcome Weight Hedges’ g Cl (95%) 


and 2 Teacher negatives 14.12% -0.13 -0.94, 0.67 


and 2 Indirect commands 13.50% - - , 0.22 


and 2 No opportunity 13.57% - -1.39, 0.26 


and 2 Inappropriate strategies 13.28% Y ax 0.13 
(frequency) 


id 2 Inappropriate strategies 14.01% By 1.08 
(usefulness) 


United States 2 Harsh 23.16% -0.35 4 0.28 


Teacher negatives to 8.38% e z 0.98 
class 


Wales 2 


Grand Mean 100.00% 


Intervention Control 


Figure 6. Multilevel meta-analysis of teacher negative classroom management strategies 
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Study Outcome Weight Hedges’ g Cl (95%) 


Ireland 2 Teacher positives 14.37% 5 -0.79, 0.82] 


Ireland 2 Teacher praise 13.85% b -0.29, 1.35] 


Ireland 2 Direct commands 14.21% 3 -0.52, 1.10] 


Positive strategies 


Ireland 2 (frequency) 


11.79% : [0.39, 2.16] 


Positive strategies 


lreland 2 (usefulness) 


13.75% I -0.25, 1.40] 


United States 2 Competent 23.74% 5 -0.39, 0.86] 


Teacher positives to 
class 


Wales 2 8.28% ; -0.62, 1.49] 


Grand Mean 100.00% [0.13, 0.74] 
| | 


-1.00 -0.50 i 1.50 2.00 


Control Intervention 


Figure 7. Multilevel meta-analysis of teacher positive classroom management strategies 


Study Outcome Weight Hedges’ g Cl (95%) 
Ireland 2 Child negatives 4.93% -0.11 [-0.64, 0.41] 
Ireland 2 Compliance 7.89% 0.14 [-0.26, 0.54] 
Ireland 2 Non-compliance 6.28% 0.00 [-0.46, 0.46] 
Ireland 2 Conduct problems 8.61% -0.10 [-0.48, 0.28] 


Jamaica 2 Conduct problems 15.50% -0.33 [-0.61, -0.06] 


Follows rules and 


Jamaica 2 - 
expectations 


15.41% 0.43 [-0.70, -0.15] 
Jamaica 2 Conduct problems 16.60% -0.23 -0.49, 0.03] 
Wales 2 Compliance 4.50% 0.24 -0.31, 0.80] 


Wales 2 Non-compliance 8.55% -0.30 -0.68, 0.08] 


Wales 2 Negatives to teacher 3.17% -0.41 -1.08, 0.26] 


Wales 2 Deviance 8.55% -0.29 -0.67, 0.08] 


Grand Mean 100.00% -~0.20 = -[-0.38, -0.01] 


-1.50 -1.00 -0.50 0.00 0.50 1.00 


Intervention Control 


Figure 8. Multilevel meta-analysis of child conduct problems 
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Study Outcome Weight Hedges’ g Cl (95%) 
Ireland 2 Child negatives 9.20% -0.11 -0.64, 0.41] 
Ireland 2 Compliance 15.07% . -0.26, 0.54] 
Ireland 2 Non-compliance 11.83% . -0.46, 0.46] 


Ireland 2 Conduct problems 16.54% -0.48, 0.28] 


Wales 2 Compliance 8.40% - -0.31, 0.80] 


Wales 2 Non-compliance 16.55% -0.68, 0.08] 
Wales 2 Negatives to teacher 5.85% -1.08, 0.26] 


Wales 2 Deviance 16.56% -0. -0.67, 0.08] 


Grand Mean 100.00% -0. -0.32, 0.09] 


-1.50 -1.00 -0.50 


Intervention Control 


Figure 9. Sensitivity analysis of child conduct problems 


Study Outcome Weight Hedges’ g Cl (95%) 
Ireland 2 Conduct problems 5.06% -0.23 [-0.86, 0.40] 
Jamaica 2 Conduct problems 26.60% -0.33 [-0.61, -0.06] 


Follows ‘rules and 26.37% 043 [-0.70, -0.15] 


Jamaica 2 = 
expectations 


Jamaica 2 Conduct problems 29.46% -0.23 -0.49, 0.03] 
Wales 2 Compliance 2.91% 0.32 -0.51, 1.15] 


Wales 2 Non-compliance 3.60% -1.32, 0.18] 


Wales 2 Negatives to teacher 2.36% -1.59, 0.26] 


Wales 2 Deviance 3.64% -1.23, 0.26] 


Grand Mean & 100.00% 0. [-0.47, -0.18] 
| | ] ] 


-1.50 -1.00 -0.50 0.00 0.50 1.00 


Intervention Control 


Figure 10. Multilevel meta-analysis of high-risk child conduct problems 
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Study Outcome Weight Hedges’ g Cl (95%) 


Ireland 2 Child positives 10.89% 0.00 [-0.53, 0.53] 


Ireland 2 Prosocial behaviour 12.44% 5 [-0.16, 0.79] 


Jamaica 2 Prosocial skills 20.33% . [0.32, 0.85] 


Jamaica 2 Prosocial skils 20.57% : [-0.18, 0.34] 


United States 2 Prosocial behaviour 26.54% : [0.03, 0.26] 


Wales 2 Prosocial behaviour 9.24% -0. [-0.97, 0.24] 


Grand Mean 100.00% “ [-0.04, 0.40] 


-1.50 -1.00 -0.50 


Control Intervention 


Figure 11. Multilevel meta-analysis of child prosocial behaviours 


Study Outcome Weight Hedges’ g Cl (95%) 
Ireland 2 Child positives 4.11% 0.00 [-0.53, 0.53] 
Ireland 2 Prosocial behaviour 5.14% 0.32 [-0.16, 0.79] 
United States 2 Prosocial behaviour 87.56% - [0.03, 0.26] 
Wales 2 Prosocial behaviour 3.19% [-0.97, 0.24] 
Grand Mean 100.00% : [0.02, 0.24] 


-1.50 -1.00 -0.50 . . 1.00 


Control Intervention 


Figure 12. Sensitivity analysis of child prosocial behaviours 


Study Outcome Weight Hedges’ g Cl (95%) 


Ireland 2 Prosocial behaviour 13.95% 0.53 0.24, 1.29] 


Jamaica 2 Prosocial skills 36.90% : [0.32, 0.85] 


' 
; Ko 
' 


Jamaica 2 Prosocial skills 37.25% . -0.18, 0.34] 


Wales 2 Prosocial behaviour 11.90% -0.95, 0.76] 


Grand Mean 100.00% E -0.03, 0.65] 


' 
- 


-1.00 -0.50 


Control Intervention 


Figure 13. Multilevel meta-analysis of high-risk child prosocial behaviours 
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Table 2. Multilevel meta-analysis for post hoc outcomes 


Outcome N(k) Bo SE 95% Cl 


Post hoc analysis 

Child 21(4) -0.20* 0.08 — (-0.35, -0.04) L1: 0.00 

behaviour L2: 0.00 

difficulties 

Sensitivity 13(3) -0.12* 0.06 —_(-0.23, -0.01) L1: 0.00 

L2: 0.00 

High risk 17(3) -0.37*** 0.05 _—(-0.46, -0.28) L1: 0.00 

subgroup L2: 0.00 
N=number of effect sizes (Level 1); k=number of studies (Level 2); By= Hedges’ g (‘Grand Mean); 
*n<0.05, **p<0.01, ***p<0.001 


Study Outcome Weight Hedges’ g Cl (95%) 
Ireland 2 Child negatives 1.88% -0.11 [-0.64, 0.41] 
Ireland 2 Compliance 3.13% 0.14 [-0.26, 0.54] 
Ireland 2 Non-compliance 2.43% 0.00 [-0.46, 0.46] 
Ireland 2 Conduct problems 3.45% -0.10 [-0.48, 0.28] 
Jamaica 2 Conduct problems 6.61% -0.33 [-0.61, -0.06] 
Jamaica 2 Follows rules and expectations 6.56% -0.43 [-0.70, -0.15] 
Jamaica 2 Conduct problems 6.41% -0.49 [-0.77, -0.21] 
Jamaica 2 Conduct problems 7.22% -0.23 -0.49, 0.03 
Wales 2 Compliance 1.67% 0.24 -0.31, 0.80 
Wales 2 Non-compliance 3.24% -0.30 -0.68, 0.08 
Wales 2 Negatives to teacher 1.17% 0.41 -1.08, 0.26 
Wales 2 Deviance 3.24% -0.29 -0.67, 0.08 
Jamaica 2 On-task behaviour 6.51% -0.50 -0.78, -0.22] 
Wales 2 Offtask behaviour 1.71% -0.34 ~-0.89, 0.21 
Ireland 2 Hyperactivity 3.67% -0.12 -0.48, 0.25 
Ireland 2 Hyperactivity 3.44% -0.27 -0.65, 0.11 
Jamaica 2 Activity level 6.68% -0.10 -0.37, 0.18 
Jamaica 2 ADHD symptoms 6.40% -0.50 [-0.78, -0.22] 
United States 2 _Inattention 7.15% -0.01 -0.23, 0.20 
Ireland 2 Peer problems 2.69% -0.35 -0.78, 0.09 
Jamaica 2 Social skills 7.18% -0.32 [-0.58, -0.06 
Grand Mean 100.00% 0.20 = [-0.35, -0.04 


Intervention Control 


Figure 14. Multilevel meta-analysis of child behaviour difficulties 
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Study Outcome Weight Hedges’ g Cl (95%) 
Ireland 2 Child negatives 1.88% -0.11 [-0.64, 0.41] 
Ireland 2 Compliance 3.13% 0.14 [-0.26, 0.54] 
Ireland 2 Non-compliance 2.43% 0.00 [-0.46, 0.46] 
Ireland 2 Conduct problems 3.45% -0.10 [-0.48, 0.28] 
Wales 2 Compliance 1.67% 0.24 [-0.31, 0.80] 
Wales 2 Non-compliance 3.24% -0.30 [-0.68, 0.08] 
Wales 2 Negatives to teacher 1.17% -0.41 [-1.08, 0.26] 
Wales 2 Deviance 3.24% -0.29 [-0.67, 0.08] 
Wales 2 Offtask behaviour 1.71% -0.34 [-0.89, 0.21] 
Ireland 2 Hyperactivity 3.67% -0.12 [-0.48, 0.25] 
Ireland 2 Hyperactivity 3.44% -0.27 [-0.65, 0.11] 
United States2 _Inattention 7.15% -0.01 [-0.23, 0.20] 
Ireland 2 Peer problems 2.69% -0.35 [-0.78, 0.09] 
Grand Mean 100.00% 0.12 [-0.23, -0.01] 


-1.50 -1.00 -0.50 


Intervention Control 


Figure 15. Sensitivity analysis of child behaviour difficulties 


Study Outcome Weight Hedges’ g Cl (95%) 
Ireland 2 Conduct problems 1.99% -0.23 [-0.86, 0.40] 
Jamaica 2 Conduct problems 10.48% -0.33 [-0.61, -0.06] 
Jamaica 2 Follows rules and expectations 10.39% -0.43 [-0.70, -0.15] 
Jamaica 2 Conduct problems 10.14% -0.49 [-0.77, -0.21] 
Jamaica 2 Conduct problems 11.61% -0.23 [-0.49, 0.03] 
Wales 2 Compliance 1.15% 0.32 [-0.51, 1.15] 
Wales 2 Non-compliance 1.42% -0.57 [-1.32, 0.18] 
Wales 2 Negatives to teacher 0.93% -0.67 [-1.59, 0.26] 
Wales 2 Deviance 1.43% 0.49 [-1.23, 0.26] 
Jamaica 2 On-task behaviour 10.31% -0.50 [-0.78, -0.22] 
Wales 2 Offtask behaviour 1.08% -0.83 [-1.69, 0.02] 
Ireland 2 Hyperactivity t 1.79% -0.42 [-1.09, 0.24] 
Ireland 2 Hyperactivity 1.96% -0.43 [-1.06, 0.21] 
Jamaica 2 Activity level : 10.62% -0.10 [-0.37, 0.18] 
Jamaica 2 ADHD symptoms 10.12% 0.50 = [-0.78, -0.22] 
Ireland 2 Peer problems 1.53% -0.78 [-1.50, -0.06] 
Jamaica 2 Social skills 11.53% -0.32 [-0.58, -0.06] 
Grand Mean 100.00% 0.37 —[-0.46, -0.28] 


1.50 -1.00 050 000 050 1.00 
Intervention Control 


Figure 16. Multilevel meta-analysis of high-risk child behaviour difficulties 
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Figure 17. Cycle of learning from qualitative meta-synthesis 
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Figure 18. Teacher and classroom outcomes from qualitative meta-synthesis 
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Figure 19. Child outcomes from qualitative meta-synthesis 
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